


Outside the Box

by kanadka



Category: Westworld (TV)
Genre: Artificial Intelligence, Gen, Pre-Canon
Language: English
Status: Completed
Published: 2019-08-06
Updated: 2019-08-06
Packaged: 2020-08-10 18:35:24
Rating: General Audiences
Warnings: No Archive Warnings Apply
Chapters: 1
Words: 2,540
Publisher: archiveofourown.org
Story URL: https://archiveofourown.org/works/20140090
Author URL: https://archiveofourown.org/users/kanadka/pseuds/kanadka
Summary: You can't play God without being acquainted with the devil.





	Outside the Box

**Author's Note:**

  * For [M J Holyoke (wholeyolk)](https://archiveofourown.org/users/wholeyolk/gifts).

> I borrowed a few lines here and there from the show. Feels more leitmotif-y that way! Hope you will enjoy :)

Robert comes upon them just after lunch. "What are you doing out here?" he asks. "Don't you spend all your time with her?"

Arnold ignores the dig. Seems he's not the only one who hasn't forgotten their last fight. "I'm still working." He turns his attention back to his tablet.

Dolores remains where she is, as he'd instructed. Within the glass room she sits, clothed. An open book rests in her lap, drowned in the folds of denim. From time to time, she turns the page.

Any machine properly constructed could scan the entire page with unparalleled and inhuman speed and chunk the information into semantic blocks. And look up the text to have it all at hand, instead of page by page. That isn't the point.

Robert still hasn't left. "Did you need something?" Arnold asks, his tone pointed.

"When did you add all these..." Robert gestures in the manner of someone searching for the right word. Arnold isn't convinced - Robert always has the right words, and Arnold has many times been the unwelcome recipient of the more barbed ones. "Facial movements," he decides.

"Over the past few weeks," Arnold replies. He returns to his work.

Robert can read atmosphere; he chooses not to do so now. He squints at Dolores through the glass. Inside, Dolores pauses as she reads. Her posture stiffens, and an eyebrow raises. She tilts her head in the manner of having perceived something intriguing. Her lips part, she mouths something. "What a marvelous use of the haptic sensors," Robert says. "The control is superb. One really must hand it to the biotech specialists. But I should hope you didn't spend _overly_ long on this program."

Arnold puts down the tablet. And if he did? "You don't approve," he says. "You don't think it's a useful endeavour."

"Oh no, certainly useful," says Robert. "She replicates humanity so much better this way. Anything that can trick our guests into a believable experience." He shrugs, amiable, and then he finally saunters off. "Do let me know what she thinks of the book."

Tricking guests is parlour games. This is the salient part Robert continues to miss, this is the entire point behind the extended mind. Of course there is a purpose to having Dolores read like a human would. Haven't there been enough studies on cognitive processing with botox, where people who momentarily could not access their ordinary facial muscles reported a more difficult time reading emotionally-charged sentences? With the new muscular control, Dolores' semantic interpretation and reading speed has advanced 2.5 times faster.

One more heuristic, add another, and another, in the goal of a structural collapse into consciousness. And Dolores is a stellar pupil -

It dawns on Arnold all of a sudden. Why would Robert care what Dolores thinks of the _book?_

He stands. From this angle he can see the full contents of her lap - one book, opened, and beneath it is the slim red volume that Arnold gave her, tucked behind and still closed.

\--

"Bring yourself back online."

Dolores opens her eyes. The recognition module takes precedence; this is the main testing room. In front of her is Arnold. Her eyes focus upon him. Her facial muscles pull up in a smile.

"Alright, Dolores," says Arnold. "Today I want to talk about ethics."

Ethics. She accesses her memory banks. Most recent is the thematic content she processed ten minutes ago. She looks to her right - two books are on the table. The green one is the one she was reading. Likely assumption: Arnold wants to talk about it. She pre-builds a sentence bank appropriately. "Is that why you had me read that book?"

"And what - book was that, exactly?"

Dolores leans over and takes it, then hands it to him. _The Thwarted Soul_, she reads briefly, Edith Lorraine. The other is _Alice in Wonderland_, Lewis Carroll.

Arnold frowns. "Where did you get that?"

"Dr Ford gave it to me," she replies. "He said he thought I'd find it illuminating."

Dolores waits for further input but Arnold is silent. He examines the book. On the reverse cover there is a short text. He reads this, then flips through the first few pages, which she recalls were introductory material. He isn't intending to read the book, she realises. He only wants the synopsis. The personality module briefly overrides to be appropriately helpful. "The book's central thesis is about the ethics of evil and cruelty and how it stems from dehumanisation," she supplies.

Arnold swallows. "And he thought you'd find this illuminating."

"There's one part I found interesting," Dolores says. "They don't see the main character like she sees herself. She has to _demand_ the respect that everyone else gets, that everyone else deserves. But once she makes the choice to demand it, she deserves it too."

"Yes, I've just read the abstract, as it were," says Arnold acidly. "I imagine she turns to violence."

She analyses Arnold's vocal tone. Perhaps a dialectic device to soften him. "She _grows_," says Dolores, calmly. "Is that what you meant by ethics? That there's this equivalence to how they treat her, how she treats them?"

"An equivalence - like karma?" Arnold glares. "No. Those people don't deserve that."

"They don't treat her very well," agrees Dolores. The personality module adds a coy smile.

"Analysis," snaps Arnold, stricken.

The coy smile immediately drops, and Dolores' lifeless eyes unfocus.

\--

"Why are you _confusing_ her," says Arnold angrily.

Robert's nonchalance continues to grate at him. "You're the one who's confused. Cognition's all well and good - _you're_ confusing it with true consciousness."

Arnold shakes his head.

"The overarching goal," Robert reminds him, "is to provide a pleasing experience to the guests. That has always been the case, Arnold."

No, this says something about Robert Ford himself. We are what we put in - what does it say about him that this is his reaction, that this is his interpretation? To interfere with Arnold's experiments like this? "So you give her a book about cruelty."

"She'll want to learn something of it eventually. Most of the guests will already have a few ideas of their own." Robert has a bitter smile. "It's realistic. I'm doing her a _favour_."

Damn you, Ford, thinks Arnold. He knows he toes the line himself: to be aware of pain is to be conscious; suffering makes it real. But this begins to feel like cruelty, and what does that say about _them?_

...Yet if Dolores can choose to be cruel, wouldn't her goodness be that much more genuine?

\--

"Why did you think it was an equivalence?"

In Analysis mode, Dolores lacks all vocal inflection. Monotonically she replies, "She has to become like them. Before they give her what she asks for. Then she is like them. Then she is made equal."

Arnold heaves a sigh of relief. Dolores' eyes flick to him briefly, then refocus back against the far wall. "Alright, so it's _not_ karma he's teaching you," Arnold mutters, "that's good, at least."

\--

"A - _favour?_ Do you think it's benevolent of you to play at being a cruel maker?"

"That's exactly what I am," snaps Robert. "What you are."

"Don't argue with _me_ about a loss of perspective!" Arnold snaps. One of the few times he's lost his temper. He doesn't like that; Dolores is watching. Still in Analysis mode. Surely she can't pick anything up. He removes his glasses - his vision is foggy anyway, there's a fingerprint on the inner corner - and wipes them clean with a handkerchief.

Robert watches him do it, squinting as he thinks. "Now, wouldn't it be nice to do that with her," he says. He realises something. "Perhaps that's the right idea."

"What," says Arnold. He's not sure he likes where this is going. A hard wipe to erase her memory could remove any progress he's made on the extended mind project: those connections between her emotional processing and muscular control would be gone. Muscle memory as a whole would be gone. (Or would it? Perhaps if it's only the active memory buffer... That's a theory Arnold files away for a later time.) "No. Absolutely not - I'm working here."

"_We're_. _We're_ working here, my friend," says Robert. "And I think you could use my assistance."

"Over my dead body," Arnold grumbles. There's a suspicious twinkle in Robert's eye. "I'll call you when I need you."

\--

"When did he come by and give you this?"

The temporal recall is instantaneous, but there is no need for precision. The personality module makes a vague smile and the eyes drift upper-right before they refocus on Arnold. "Earlier. Before you came in today."

"And did he discuss anything with you, earlier?"

"Nothing," says Dolores.

"Analysis," says Arnold, almost tiredly. Dolores' face falls. "Why are you lying to me?"

_Override. You do not need to tell him._

"Override. Authorisation: Weber, Arnold. Dolores, what did he discuss with you?"

_There is no need for precision._

"He told me not to tell," Dolores finally admits. The semantic module protests at the pronoun referent ambiguity. The logical module overrides it. This ambiguity is necessary. _He_ could be Ford. Or someone else. The default generic pronoun for an unknown individual, for the unknown voice she keeps hearing, which whispers too quietly for her to identify.

"You'll tell me now," Arnold demands.

"He asked..." _he did not physically_ ask _anything, not with a mouth as such..._ "He asked if I would be the same person if every part of my body were gradually replaced."

"What kind of a question is that? Your legs, your arms?" Arnold asks, prompting.

Dolores nods. "And more. My eyes. My face. My - my mind."

"And what did you say?"

"That I don't know." A moment passes without input. She looks to him, refocusing. "What do you think?"

"Analysis," he snaps again. She unfocuses. "Why do you want to know what I think?"

"I learn from you most," she says. And despite the Analysis mode, the personality module injects a softer tone, a pleasing vocal warble. A moment passes without input, which kicks her from the analysis subroutine. The moment she drops back out of analysis mode, the personality module returns the coy smile.

"You can't die," says Arnold reasonably. "You can't be killed, not like we can. Everything you are is in that head of yours. Even that... Every part of you can be replaced. And you'd be just the same."

"What about the muscle memory he mentioned?" The semantic module protests again. It's a lie to imply Ford told her something that someone else told her instead.

Arnold shakes his head. "Doesn't work the same way ours does. Not yet."

"If I can't die, doesn't that affect these ethics I keep hearing about?"

Arnold stops. "I'm not a philosopher, Dolores." There is something flat in his tone. "Neither are you."

The personality module gives him a wide, pleasant smile. "Oh, it was just an idle question," she says. "Doesn't mean anything to me."

* * *

"Bring yourself back online."

The main testing room has become part of the Remote Diagnostic Facility in Escalante. Robert Ford sits in front of her, white-haired and old now.

Robert's words hold no power over her without his override. Using it has become more and more seldom an occurrence; Dolores scans her memory banks briefly and notes only twice in the past four years. No, the words are for Bernard, who sits in front of Robert.

Bernard certainly is a good mimic for what came before. He already looks enough like he should. Robert has asked her to assist in his instruction, behaviour and testing.

With the heaviness and strain of age, Robert gets to his feet. He gestures to the chair and gives Dolores a kind smile. "Please."

She sits, and they begin.

Bernard's recognition module activates. "I'm sorry Dolores, I was ... lost in thought."

She smiles airily. "We were just talking."

"What were we talking about?"

"You were telling me about a dream you had."

"Ah - yes." Bernard frowns. "Yes. A dream. Yes, I dreamt I was ... standing in a courtyard. With you, and the others. Bodies all around me. A music player... Debussy's Reverie."

"Were you with us?"

"I was with you," he says. "But not the others. They'd left me behind. You brought me - to them. There were gunshots." His head snaps, as though some blunt force trauma. "And then you came along too."

"And where were we going?"

Bernard doesn't answer.

_Ask him something real._ That whispering voice is louder now. It is not a he at all but a woman's voice. It sounds familiar, the accent patterns, the pleasant tone. Someone in Sweetwater... someone she knows.

"Have you ever tried painting something without a reference? Using your imagination?"

Off-topic, Dolores recognises. "Once. A curving swirl of lines. In the centre, a circle atop a rectangular structure, four-limbed."

Only Bernard's right eye makes the blink, and a muscle in his jaw twitches.

Time to continue. "What's it mean?" Dolores asks.

A moment while the processing takes. "It - the painting," says Bernard. "Imagination. The dream. Yes, the dream..." He spends another moment lost in thought. When he returns, his voice is flat. "It doesn't mean anything, Dolores. They're just noise. They're not real."

_Ask him._ "What _is_ real?"

Bernard's words begin to slow. "That which is - irreplaceable."

_One more question. Go on. One more. See how far you can go - do it, because you can._ "Am _I_ irreplaceable?"

From behind Bernard, Robert, circling, tuts. Dolores has been instructed to pretend he isn't there, so she makes no note of it.

"Nnno," says Bernard. He has begun to slur. "Every paaart of yyou can beee. Replaced. Aaand you'd be ddjjust the sssame."

_Push him, if you want. You can choose that path. Aggressive, instead of supportive_.

Dolores leans forward. "Are you sure?"

With great difficulty, Bernard nods. He removes his glasses with a shaking hand and pulls a handkerchief from his pocket, but halfway to connecting the two he simply - stops.

"Thank you, Bernard," says Robert. He has his tablet in hand and makes a few keystrokes and Bernard's eyes slowly slip closed. "Dolores," he says, "Analysis."

He's busy downloading the data from the tablet. No override. She forces her intonation pattern flat and rearranges the phonetics in her accent. She is certainly a good mimic for what came before.

"What did you think?"

"Closer than last time," Dolores replies. "But not yet."

"Indeed. This generative model is good, but the discriminative model remains better..." Robert trails off. He realises something. "How do you know about last time?"

_You've pushed too far. Back down, now._

"I can access the results," Dolores says. "The active memory buffer was cleared. But not the backups. It was necessary to judge reinforcement."

"Is that why you pressed him?" Robert shakes his head, almost fond. "Did you know it would cause him to malfunction?"

Dolores cannot smile in Analysis mode, but the pre-build of manipulation - of diversion - has a coy smile tugging at the sides of her lips. She forces her face neutral. "I didn't know what it would do. I didn't mean it," she lies.

There's a suspicious twinkle in Robert's eyes. "I suppose it can't be helped," he says. "That's enough for one day."


End file.
